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DETAILED ACTION 
Claim Rejections - 35 USC § 101 

1) 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

1-1 Claims 1-42 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. 

Claims 1-42 set forth non-functional descriptive material but fail to set forth 
physical structures or materials comprising of hardware or a combination of hardware 
and software within the technological arts (ie., computer) to produce a "useful, concrete 
and tangible" result. For example, Claims 1,17 and 22 reads on a mental 
construct/abstract idea or at best a computer program, per se. The language 
"visualizing digital media", etc., does not clearly define structural elements and are not 
tangibly embodied on a computer readable medium. Additionally, claim 16 reads on a 
mental construct/abstract idea or at best a computer program per se. The language " 
computing the size of a new element to placed in a weighted presentation grid", etc., 
does not clearly define structural elements and are not tangibly embodied on a 
computer readable medium. 

Claim Rejections - 35 USC §112 

2) The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 



' Application/Control Number: 09/784,369 
Art Unit: 2176 

2- 1 ) Claim 16 recites the limitation "significance weight" in paragraph (b) 
insufficient antecedent basis for this limitation in the claim. 

Claim Rejections - 35 USC § 102 

3) The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

3- 1) Claims 1,2,3,4,5,6,8,9,10, 17, 18, 19, 22,23, 24 25, 26, 27, 39, 42 are rejected 
under 35 U.S.C. 102(b) as being unpatentable over Kerne publication, 
CollageMachine: Temporality and Indeterminacy in Media Browsing via Interface 
Ecology (Marcy 22-27, 1997, as provided in the applicant's IDS, copyrighted 1997 
by Andruid Kerne). 

For Claim 1, Kerne teaches "a presentation grid ... document" (ie., collage layout 
based on user preference)(page 1)(ie., image URL downloaded - new Netscape 
browser window)(page 3). 

Kerne teaches "an accessor .... documents" (ie., media pulled from web 
pages)(page 1 - Abstract section). 

Kerne teaches 'a document parser ... document components' (ie., parsing of 
HTML ... first HTML file ... partially parsed)(page 3). 



Page3 
. There is 




■Application/Control Number: 09/784,369 Page 4 

Art Unit: 2176 

Kerne teaches 'an agent to store ... said model' (ie., relational database as a 
source for media; user-responsive performance)(page 2)(ie., user's expression of 
preferences „. retrieve)(page 1). 

For Claim 2, Kerne teaches " ... response to user input" (ie., user clicks)(page 

3). 

For Claim 3, Kerne teaches " document component :.. model" (ie., relational 
database)(page 2); 

For claim 4, Kerne teaches "a weight ... generated by the document parser" (ie., 
mechanism for expressing interest or disinterest in collage elements... prioritizing)(page 
3). 

Kerne teaches "alter ... user input" (ie., element may be brought to the top or 
removed)(page 3). 

Kerne teaches "document component ... said model" (ie., user clicks ... 
displaying media elements on the layout)(page 3). 

For claim 5, Kerne teaches "at least one ... in said model" (ie., user interest, 
disinterest ... prioritizing media elements)(page 3). 

For claim 6, Kerne teaches "at least one ... said presentation grid" (ie., elements 
brought to top or removed from collage)(page 3). 

For claim 8, Kerne teaches "... acquire HTML pages" (ie., HTML file)(page 3). 

For claim 9, Kerne teaches "... multimedia documents" (ie., media elements are 
pulled)(page 1). 

For claim 10, Kerne teaches "... Internet" (ie., the web)(page 1 ). 
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For claim 17, Kerne teaches "providing ... medial elements" (ie., ... media 
elements)(Abstract section, page 1). 

Kerne teaches "assigning ... media element" (ie., user interest, disinterest ... 
prioritizing media elements)(page 3). 

Kerne teaches "establishing a presentation grid" (ie.,CollageMachine has a 
collage layout; element may be brought to the top or removed)(page 1 -3). 

. Kerne teaches "selecting a media element ... significance weights," "placing ... 
grid" (ie., user clicks ... displaying media elements on the layout)(pages 1-3). 

For claim 18, Kerne teaches "providing the initial value " (ie., user's expression 
of preferences and interests with design rules and semantic rules to make decisions 
about the collage layout and about which media to retrieve)(page 3). Specifically, upon 
the user's input of the parameters, the initial values to the prioritization is set. 

Kerne teaches determining if any ... increasing the significance ... response" (ie., 
mechanism for expressing interest or disinterest in collage elements... prioritizing)(page 
3). 

For claim 19, Kerne teaches "increasing ... to user input" (ie., mechanism for 
expressing interest or disinterest in collage elements... prioritizing)(page 3). 

For claim 22, Kerne teaches "decomposing ... components" (ie., parsing HTML 
files into chunks and image URLs)(page 3). 

Kerne teaches "assigning ... components" (ie., expressing interest/disinterest 
...)(page3). 
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Kerne teaches "storing ... document references" (ie., relational database)(page 

2). 

' Kerne teaches "establishing ... grid" (ie., collageMachine layout, temporal 
composition)(page 2). 

Kerne teaches " selecting ... weights (ie., priortization)(page 3). 

Kerne teaches " placing ... grid" (ie., collageMachine ... visual assemblage)(page 

1-3). 

. For claim 23, Kerne teaches "assigning ... element " (ie., user interest, 
disinterest ... prioritizing media elements)(page 3). 

For claim 24, Kerne teaches "parsing ... tags" (ie., parsing HTML file, 
multithreaded Java implementation... )(page 3, top). 

For claim 25, "... HTML tags " (ie., HTML ...)(page 3, top). 

For claim 26, Kerne teaches "providing ... weight" (ie., mechanism for 
expressing interest or disinterest in collage elements... prioritizing)(page 3). 

Kerne teaches "determining if any of said ... in response " (ie., GUI for user to 
express interest or disinterest ... brought to top or removed ... prioritizing)(page 3, GUI 
section). 

For claim 27, Kerne teaches "increasing ... user input" (ie., GUI for user to 
express interest or disinterest ... brought to top or removed ... prioritizing)(page 3, GUI 
section). 
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For claim 39, Kerne teaches "selecting a ... operation" (ie., prioritizing the 
media elements upon the user expressing interest and bringing to top or removing 
elements from collage)(page 3, middle). 

For claim 42, Kerne teaches "selecting ... operation"(ie., prioritizing media 
elements based on user interest or disinterest)(page 3, GUI section). 

Claim Rejections - 35 USC § 103 
4) The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4-1 ) Claims 7, 1 1, 12, 1 3, 14, 15, 20, 28, 34 , 35, 36, 37, 38, 40, 41 are rejected 

under 35 U.S.C. 103(a) as being unpatentable over Kerne fas cited above), in view 

of Perlin et al (US 5342466, issued Aug 1994, as cited in the applicant's IDS). 

For claim 7, Kerne does not expressly teach, but Perlin teaches "... floating point 
number" (ie., ... floating point number)(col 17, lines 5-15). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach a floating point number as taught by Perlin, providing 
the benefit of a computer user interface that represents information stored in a computer 
as representation objects located on at least one reference surface, where images in 
the sequence is a bit-mapped representation of an object (Perlin, Abstract), and a 
fractal interface capabilities with internal bit-mapped representations of images to the 
workstation screen (Perlin, col 8, lines 40-45). 
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For claim 1 1 , Kerne does not expressly teach, but Perlin teaches "a doubly 
linked list ... said presentation grid" (ie., doubly linked list)(col 10, lines 50-55; fig 6, item 
20). " 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach a doubly linked list as taught by Perlin, providing the 
benefit of a computer user interface that represents information stored in a computer as 
representation objects located on at least one reference surface, where images in the 
sequence is a bit-mapped representation of an object (Perlin, Abstract), and a fractal 
interface capabilities with internal bit-mapped representations of images to the 
workstation screen (Perlin, col 8, lines 40-45). 

For claim 12, Kerne does not expressly teach, but Perlin teaches "an array ... 
rapid sorting ... presentation grid" (ie., images stored as files in the array ...)(col 1 1 , 
lines 22-40). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach images stored as files in the array as taught by 
Perlin, providing the benefit of a computer user interface that represents information 
stored in a computer as representation objects located on at least one reference 
surface, where images in the sequence is a bit-mapped representation of an object 
(Perlin, Abstract), and a fractal interface capabilities with internal bit-mapped 
representations of images to the workstation screen (Perlin, col 8, lines 40-45). 
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For claim 13, Kerne does not expressly teach, but Perlin teaches " placement 
grid ... placement are reduced "(ie M two-dimensional coordinate system defined for 
each surface)(co| 7, lines 45-68). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach a two-dimensional coordinate system as taught by 
Perlin, providing the benefit of a computer user interface that represents information 
stored in a computer as representation objects located on at least one reference 
surface, where images in the sequence is a bit-mapped representation of an object . 
(Perlin, Abstract), and a fractal interface capabilities with internal bit-mapped 
representations of images to the workstation screen (Perlin, col 8, lines 40-45). 

For claim 14, Kerne does not teach, but Perlin teaches "two dimensional array of 
grid cell objects ... object boundary" (ie., two-dimensional coordinate system defined for 
each surface; bounding rectangle)(col 7, lines 45-68; col 1 1 , lines 1-20). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach a two-dimensional coordinate system and bounding 
rectangle on the document component for aligning as taught by Perlin, providing the 
benefit of a computer user interface that represents information stored in a computer as 
representation objects located on at least one reference surface, where images in the 
sequence is a bit-mapped representation of an object (Perlin, Abstract), and a fractal 
interface capabilities with internal bit-mapped representations of images to the 
workstation screen (Perlin, col 8, lines 40-45). 
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For claim 15, Kerne does not expressly teach, but Perlin teaches "a doubly 
linked list ... each grid cell object" (ie., doubly linked list)(col 10, lines 50-55; fig 6, item 
20). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach a doubly linked list as taught by Perlin, providing the 
benefit of a computer user interface that represents information stored in a computer as 
representation objects located on at least one reference surface, where images in the 
sequence is a bit-mapped representation of an object (Perlin, Abstract), and a fractal 
interface capabilities with internal bit-mapped representations of images to the 
workstation screen (Perlin, col 8, lines 40-45). 

For claim 20, Kerne does not teach, but Perlin teaches "creating a ... cells," 
"establishing ... media elements" (ie., doubly linked list)(col 10, lines 50-65; fig 6, item 
20). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach a doubly linked list for media elements as taught by 
Perlin, providing the benefit of a computer user interface that represents information 
stored in a computer as representation objects located on at least one reference 
surface, where images in the sequence is a bit-mapped representation of an object 
(Perlin, Abstract), and a fractal interface capabilities with internal bit-mapped 
representations of images to the workstation screen (Perlin, col 8, lines 40-45). 
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For claim 28, Kerne does not teach, but Perlin teaches "creating a ... cells," 
"establishing ... media elements" (ie., doubly linked list)(col 10, lines 50-65; fig 6, item 
20). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach a doubly linked list for media elements as taught by 
Perlin, providing the benefit of a computer user interface that represents information 
stored in a computer as representation objects located on at least one reference 
surface, where images in the sequence is a bit-mapped representation of an object 
(Perlin, Abstract), and a fractal interface capabilities with internal bit-mapped 
representations of images to the workstation screen (Perlin, col 8, lines 40-45). 

For claim 30, Kerne teaches " creating ... grid cell" (ie., visual media elements 
may overlap . . . )(page 2, bottom). 

For claim 34, Kerne does not teach, but Perlin teaches "choosing ... element," 
"identify ... size" (ie., user can magnify or reduce ...)(col 3, lines 45-65). 

Kerne teaches "determining a weight ... region," "selecting ... weights" (ie., user 
clicks ... GUI ... user ... expressing interest or disinterest ... )(page 3, GUI section). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach magnifying or reducing display information as taught 
by Perlin, providing the benefit of a computer user interface that represents information 
stored in a computer as representation objects located on at least one reference 
surface, where images in the sequence is a bit-mapped representation of an object 
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(Perlin, Abstract), and a fractal interface capabilities with internal bit-mapped 
representations of images to the workstation screen (Perlin, col 8, lines 40-45). 

For claim 35, Kerne teaches "... grid region ... size" (ie., prioritizing media 
elements based on user interest)(page 3). 

For claim 36, Kerne does not teach expressly, but Perlin teaches "... assigning 
an initial size ... element" (ie., when user initially magnifies or reduces the display 
information)(col 3, lines 50-60; col 6, lines 57-58). 

Kerne does not teach expressly, but Perlin teaches "determining an ... pixel ... 
grid units" (ie., user magnifies or reduces initially)(col 3, lines 50-67; col 6, lines 57-68) 

Kerne does not teach expressly, but Perlin teaches "selecting ... pixels" (ie., pixel 
squares with images)(col 6, lines 60 - col 7, lines 5). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach magnifying or reducing display information and pixel 
squares in the bit mapped images as taught by Perlin, providing the benefit of a 
computer user interface that represents information stored in a computer as 
representation objects located on at least one reference surface, where images in the 
sequence is a bit-mapped representation of an object (Perlin, Abstract), and a fractal 
interface capabilities with internal bit-mapped representations of images to the 
workstation screen (Perlin, col 8, lines 40-45). 

For claim 37, Kerne teaches " calculating grid ... presentation grid" (ie., user 
expresses interest... elements move to top ... prioritized in CollageMachine)(page 3, 
GUI section). 
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For claim 38, Kerne teaches "calculating ... grid region" (ie., prioritizing the 
media elements upon the user expressing interest and bringing to top or removing 
elements from collage)(page 3, middle). 

For claim 40, Kerne does not expressly teach, but Perlin teaches "establishing 
... placement grid" (ie., user can magnify or reduce display information after selecting a 
region to alter... )(col 3, lines 45-68). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach magnifying or reducing display region information by 
the user as taught by Perlin, providing the benefit of a computer user interface that 
represents information stored in a computer as representation objects located on at 
least one reference surface, where images in the sequence is a bit-mapped 
representation of an object (Perlin, Abstract), and a fractal interface capabilities with 
internal bit-mapped representations of images to the workstation screen (Perlin, col 8, 
lines 40-45). 

For claim 41, Kerne teaches "calculating ... from grid cells" (ie., user expressing 
interest ... prioritizing)(page 3, middle). 

Kerne does not teach expressly, but Perlin teaches " associated screen ... 
region" (ie., user selects an area to magnify )(col 3, lines 45-68). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach magnifying or reducing display region information by 
the user as taught by Perlin, providing the benefit of a computer user interface that 
represents information stored in a computer as representation objects located on at 
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least one reference surface, where images in the sequence is a bit-mapped 
representation of an object (Perlin, Abstract), and a fractal interface capabilities with 
internal bit-mapped representations of images to the workstation screen (Perlin, col 8, 
lines 40-45). 

4-2) Claim 21, 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kerne (as cited above), in view of Pirolli et al, (US 5835905, issued Nov 1998, as 
cited in the applicant's IDS), further in view of Perlin et al (US 5342466, issued 
Aug 1994, as cited in the applicant's IDS). 

For claim 21, 29, Kerne teaches "assigning a screen weight to ... elements" (ie., 
prioritizing elements)(page 3). 

Kerne does not teach, but Pirolli does teaches "preselecting ... grid" (ie., 
prediction of need or relevance ... )(col 2, lines 1 -37). 

Kerne in view of Pirolli does not teach, but Perlin teaches "removing ... screen 
weight" (ie., displays of choices are removed)(col 1 , lines 45-57). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach prediction of need or relevance as taught by Pirolli, 
providing the benefit a large linked collection of documents to improve the rate at which 
they gain information and avoiding sluggishness to assess the global nature of the 
collection or aid in decisions about what documents to pursue (Pirolli, col 1, lines 22- 
67), and further to teach removing display choices as taught by Perlin, providing the 
benefit of a computer user interface that represents information stored in a computer as 
representation objects located on at least one reference surface, where images in the 
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sequence is a bit-mapped representation of an object (Perlin, Abstract), and a fractal 
interface capabilities with internal bit-mapped representations of images to the 
workstation screen (Perlin, col 8, lines 40-45). 

4-3) Claim 31, 32, 33 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Kerne (as cited above), in view o f Perlin et al (US 5342466, issued Aug 1994, 
as cited in the applicant's IDS), further in view of Pirolli et al, (US 5835905, issued 
Nov 1998, as cited in the applicant's IDS),. 

For claim 31, Kerne does not expressly teach, but Perlin teaches "a doubly 
linked list ... said presentation grid" (ie., doubly linked list)(col 10, lines 50-55; fig 6, item 
20). 

Kerne teaches "sorting ... screen weight" (ie., element may be brought to top or 
removed... prioritizing)(page 3). 

Kerne does not teach, but Pirolli does not teach "preselecting ... grid" (ie., 
prediction of need or relevance ... )(col 2, lines 1-37). 

Kerne in view of Pirolli does not teach, but Perlin teaches "removing ... screen 
weight" (ie. ( displays of choices are removed)(col 1, lines 45-57). 

Kerne teaches "rebuilding ... screen weight order" (ie., element may be brought 
to top or removed... prioritizing)(page 3). 

Kerne teaches "updating ... elements having lower screen weight ... user" (ie., 
element may be brought to top or removed... prioritizing)(page 3). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach a doubly linked list as taught by Perlin, providing the 
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benefit of a computer user interface that represents information stored in a computer as 
representation objects located on at least one reference surface, where images in the 
sequence is a bit-mapped representation of an object (Perlin, Abstract), and a fractal 
interface capabilities with internal bit-mapped representations of images to the 
workstation screen (Perlin, col 8, lines 40-45), further to teach prediction of need or 
relevance as taught by Pirelli, providing the benefit a large linked collection of 
documents to improve the rate at which they gain information and avoiding 
sluggishness to assess the global nature of the collection or aid in decisions about what 
documents to pursue (Pirolli, col 1 , lines 22-67). 

For claim 32, Kerne does not teach, but Perlin teaches "avoiding ... grid," (ie., 
traversing the object of the linked list ...)(col 15, lines 57 - col 16, line 15; figure 10). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach traversing a doubly linked list for media elements as 
taught by Perlin, providing the benefit of a computer user interface that represents 
information stored in a computer as representation objects located on at least one 
reference surface, where images in the sequence is a bit-mapped representation of an 
object (Perlin, Abstract), and a fractal interface capabilities with internal bit-mapped 
representations of images to the workstation screen (Perlin, col 8, lines 40-45). 

For claim 33, Kerne does not teach, but Perlin teaches "establishing a cell ... in 
the grid cell" (ie., doubly linked list for data objects of the interface ...)(col 10, lines 50- 
68). 
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Kerne does not teach, but Perlin teaches "avoiding ... grid," (ie., traversing the 
object of the linked list ...)(col 15, lines 57 - col 16, line 15; figure 10). 

Kerne does not teach, but Perlin teaches "updating ... linked list" (ie., DISPLAY 
function for copying ... parent image)(col 22, lines 41-60). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Kerne to teach traversing the objects of a doubly linked list for media 
elements as taught by Perlin, providing the benefit of a computer user interface that 
represents information stored in a computer as representation objects located on at 
least one reference surface, where images in the sequence is a bit-mapped 
representation of an object (Perlin, Abstract), and a fractal interface capabilities with 
internal bit-mapped representations of images to the workstation screen (Perlin, col 8, 
lines 40-45). 

Other Cases 

5) A. Higashio et al (US 5945982, issued Aug 1 999). 

B. Wittenberg etal (US 6515656, filed Apr 1999). 

C. Tarabella (US 5796945, issued Aug 1998). 

D. Renolds et al (US 551 3991 , issued May 1 996). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gautam Sain whose telephone number is 703-305- 
8777. The examiner can normally be reached on M-F 9-5 EST. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Feild can be reached on (703)305-9792. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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